A Neurally Plausible Encoding of Word Order Information into a Semantic Vector Space
نویسندگان
چکیده
Distributed models of lexical semantics increasingly incorporate information about word order. One influential method for encoding this information into high-dimensional spaces uses convolution to bind together vectors to form representations of numerous n-grams that a target word is a part of. The computational complexity of this method has led to the development of an alternative that uses random permutation to perform order-sensitive vector combinations. We describe a simplified form of order encoding with convolution that yields comparable performance to earlier models, and we discuss considerations of neural implementation that favor the use of the proposed encoding. We conclude that this new encoding method is a more neurally plausible alternative than its predecessors.
منابع مشابه
Encoding Sequential Information in Semantic Space Models: Comparing Holographic Reduced Representation and Random Permutation
Circular convolution and random permutation have each been proposed as neurally plausible binding operators capable of encoding sequential information in semantic memory. We perform several controlled comparisons of circular convolution and random permutation as means of encoding paired associates as well as encoding sequential information. Random permutations outperformed convolution with resp...
متن کاملA Joint Semantic Vector Representation Model for Text Clustering and Classification
Text clustering and classification are two main tasks of text mining. Feature selection plays the key role in the quality of the clustering and classification results. Although word-based features such as term frequency-inverse document frequency (TF-IDF) vectors have been widely used in different applications, their shortcoming in capturing semantic concepts of text motivated researches to use...
متن کاملPermutations as a Means to Encode Order in Word Space
We show that sequence information can be encoded into highdimensional fixed-width vectors using permutations of coordinates. Computational models of language often represent words with high-dimensional semantic vectors compiled from word-use statistics. A word’s semantic vector usually encodes the contexts in which the word appears in a large body of text but ignores word order. However, word o...
متن کاملEncoding word-order and semantic information using modular neural networks
Vector space models have been successfully used for lexical semantic representation. Some of these models rely on distributional properties of words in large corpora, and have been contrasted with human performance on semantic similarity and priming in lexical decision tasks. Neural network models of lexical representation have been classically of reduced size due to computational limitations. ...
متن کاملImproving Correlation with Human Judgments by Integrating Semantic Similarity with Second-Order Vectors
Vector space methods that measure semantic similarity and relatedness often rely on distributional information such as co–occurrence frequencies or statistical measures of association to weight the importance of particular co–occurrences. In this paper, we extend these methods by incorporating a measure of semantic similarity based on a human curated taxonomy into a second–order vector represen...
متن کامل